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MISSILE  DEFENSE 

Actions  Needed  to  Improve  Information 
for  Supporting  Future  Key  Decisions  for 
Boost  and  Ascent  Phase  Elements 


Why  GAO  Did  This  Study  What  GAO  Found 


The  Department  of  Defense  (DOD) 
has  spent  about  $107  billion  since 
the  mid-1980s  to  develop  a 
capability  to  destroy  incoming 
ballistic  missiles.  DOD  has  set  key 
decision  points  for  deciding 
whether  to  further  invest  in 
capabilities  to  destroy  missiles 
during  the  initial  phases  after 
launch.  In  March  2006,  DOD  issued 
a  report  on  these  capabilities  in 
response  to  two  mandates.  To 
satisfy  a  direction  from  the  House 
Appropriations  Committee,  GAO 
agreed  to  review  the  report. 

To  assist  Congress  in  evaluating 
DOD’s  report  and  preparing  for 
future  decisions,  GAO  studied  the 
extent  to  which  DOD  (1)  analyzed 
technical  and  operational  issues 
and  (2)  presented  complete  cost 
information.  To  do  so,  GAO 
assessed  the  report’s  methodology, 
explanation  of  assumptions  and 
their  effects  on  results,  and 
whether  DOD  followed  key 
principles  for  developing  life-cycle 
costs. 


What  GAO  Recommends 


To  support  future  decisions,  DOD 
should  include  key  stakeholders  in 
assessing  operational  issues,  report 
on  technical  progress,  and  update 
and  verify  life-cycle  cost  estimates 
in  accordance  with  key  principles 
for  developing  life-cycle  costs.  In 
comments  on  a  draft  of  this  report, 
DOD  agreed  to  include 
stakeholders  and  assess  technical 
progress  but  did  not  agree  to 
prepare  or  report  life-cycle  costs  in 
accordance  with  key  principles. 

www.gao.gov/cgi-bin/getrpt7GAO-07-430. 

To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  the  link  above. 

For  more  information,  contact  Janet  A.  St. 
Laurent  at  (202)  512-4402  or 
stlaurentj@gao.gov;  or  Paul  Francis  at  (202) 
512-2811  orfrancisp@gao.gov. 


The  report  DOD’s  Missile  Defense  Agency  (MDA)  submitted  to  Congress  in 
March  2006  included  some  useful  technical  and  operational  information  on 
boost  and  ascent  phase  capabilities  by  describing  these  elements,  listing 
upcoming  decision  points,  and  discussing  geographic  areas  where  boost  and 
ascent  elements  could  intercept  missiles  shortly  after  launch.  However,  the 
information  in  the  report  has  several  limitations  because  the  analysis  did  not 
involve  key  DOD  stakeholders  such  as  the  services  and  combatant 
commands  in  preparing  the  report  and  did  not  clearly  explain  modeling 
assumptions  and  their  effects  on  results  as  required  by  relevant  research 
standards.  MDA’s  report  states  that,  at  this  time,  some  data  is  limited,  and 
operational  concepts  that  discuss  operations  from  forward  locations  have 
not  been  fully  vetted  with  the  services  and  combatant  commands.  However, 
the  report  did  not  explain  how  each  element’s  performance  may  change  if 
developing  technologies  do  not  perform  as  expected.  Also,  it  did  not  address 
the  challenges  in  establishing  bases  at  the  locations  cited  or  provide 
information  on  the  quantity  of  each  element  required  for  various  deployment 
periods.  Moving  forward,  DOD  has  an  opportunity  to  involve  stakeholders  in 
analyzing  operational  and  technical  issues  so  that  senior  DOD  and 
congressional  leaders  will  have  more  complete  information  on  which  to  base 
upcoming  program  decisions  following  key  tests  in  2008  and  2009  for  the 
Kinetic  Energy  Interceptor  and  Airborne  Laser  boost  and  ascent  phase 
programs. 

MDA’s  report  provided  some  cost  estimates  for  developing  and  fielding 
boost  and  ascent  phase  capabilities,  but  these  estimates  have  several 
limitations  and  will  require  refinement  before  they  can  serve  as  a  basis  for 
DOD  and  congressional  decision  makers  to  compare  life-cycle  costs  for  the 
elements.  MDA’s  report  states  that  there  is  uncertainty  in  estimating  life- 
cycle  costs  because  the  elements  are  early  in  development.  However,  based 
on  a  comparison  of  the  estimates  in  the  report  with  key  principles  for 
developing  life-cycle  cost  estimates,  GAO  found  that  MDA’s  estimates  did 
not  include  all  cost  categories,  including  costs  to  establish  and  sustain 
operations  at  U.S.  bases  and  at  forward  overseas  operating  locations.  Also, 
MDA’s  estimates  did  not  calculate  costs  based  on  realistic  quantities  of  each 
element  the  combatant  commanders  or  services  would  need  to  conduct  the 
mission.  Finally,  MDA  did  not  conduct  a  sensitivity  analysis  to  assess  the 
effect  of  key  cost  drivers  on  total  costs.  MDA  officials  stated  that  further 
analysis  of  the  costs  for  each  element  along  with  measures  to  assess  their 
confidence  would  help  to  better  inform  DOD  and  congressional  decision 
makers  in  making  investment  decisions  following  key  tests  in  2008  and  2009. 


United  States  Government  Accountability  Office 


Contents 

Letter 

1 

Results  in  Brief 

4 

Background 

7 

MDA’s  March  2006  Report  Includes  Some  Useful  Information  but 

Has  Several  Limitations 

10 

DOD  Can  Significantly  Improve  Life-Cycle  Cost  Estimates  to 

Support  Future  Investment  and  Budget  Decisions  for  Boost  and 
Ascent  Phase  Capabilities 

14 

Conclusions 

18 

Recommendations  for  Executive  Action 

19 

Agency  Comments  and  Our  Evaluation 

20 

Appendix  I 

Scope  and  Methodology 

25 

Appendix  II 

Comments  from  the  Department  of  Defense 

28 

Appendix  III 

GAO  Contacts  and  Staff  Acknowledgments 

32 

Related  GAO  Products 

33 

Table 

Table  1:  Ballistic  Missile  Defense  Boost  and  Ascent  Phase 

Elements,  Key  Decision  Points,  and  Their  Planned 

Capability  Dates 

8 

Page  i 


GAO-07-430  Ballistic  Missile  Defense 


Abbreviations 


ABL 

Airborne  Laser 

BMDS 

Ballistic  Missile  Defense  System 

DOD 

Department  of  Defense 

FYDP 

Future  Years  Defense  Program 

KEI 

Kinetic  Energy  Interceptor 

MDA 

Missile  Defense  Agency 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


April  17,  2007 

Congressional  Committees: 

The  new  security  environment  includes  some  states,  such  as  North  Korea, 
that  are  aggressively  pursuing  development  of  long-range  missiles  and 
weapons  of  mass  destruction.  In  2002,  President  Bush  directed  that  the 
Department  of  Defense  (DOD)  proceed  with  plans  to  develop  and  put  in 
place  an  initial  set  of  ballistic  missile  defense  capabilities  beginning  in 
2004.  DOD  has  spent  about  $107  billion  since  the  mid-1980s  to  develop  a 
capability  to  destroy  incoming  ballistic  missiles  in  all  phases  of  their  flight, 
including  the  initial  phases  after  launch,  called  the  boost  and  ascent 
phases.1  The  purpose  of  boost  and  ascent  phase  elements  of  a  ballistic 
missile  defense  system  (BMDS)  is  to  engage  incoming  missiles  before 
release  of  warheads  or  countermeasures.  DOD  is  developing  three  BMDS 
boost  and  ascent  phase  capabilities:  Airborne  Laser  (boost  phase  only), 
Kinetic  Energy  Interceptor  (boost  and  ascent  phase),  and  the  Aegis 
Ballistic  Missile  Defense  element  that  will  fire  the  Standard  Missile-3  block 
2A2  (ascent  phase)  from  BMDS-capable  Aegis  surface  combatants. 
According  to  the  Missile  Defense  Agency  (MDA),  the  agency  responsible 
for  developing  an  integrated  ballistic  missile  defense  system,  DOD  plans  to 
spend  $1.6  billion  for  the  Airborne  Laser;  $800  million  for  the  Kinetic 
Energy  Interceptor;  and  $200  million  for  the  Aegis  Standard  Missile-3 
Block  2A  during  fiscal  years  2006-08.  The  long-term  costs  for  these 
elements  will  be  much  greater  if  DOD  continues  their  development  and 
fielding.  DOD  projects  the  initial  boost  and  ascent  phase  capabilities  will 
not  be  available  until  about  2014  through  2020.  Congress  has  raised 
questions  about  the  affordability  of  pursuing  both  the  Airborne  Laser  and 
Kinetic  Energy  Interceptor  programs. 

Over  the  next  few  years,  DOD  and  Congress  will  be  asked  to  make 
important  program  and  investment  decisions  on  boost  and  ascent  phase 
capabilities — at  a  time  when  the  federal  government  is  facing  a  large  and 


'The  boost  phase  is  the  period  immediately  after  launch  when  the  missile’s  booster  stages 
are  still  thrusting.  Next  is  the  ascent  phase  which  ends  when  the  ballistic  missile  completes 
deployment  of  reentry  vehicles  and  possible  decoys. 

"Other  versions  of  the  Standard  Missile-3  are  being  developed  to  intercept  threat  missiles  in 
their  midcourse  phase. 
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growing  fiscal  imbalance.3  Further,  we  have  identified  the  competing 
demands  of  investing  in  transformational  capabilities  while  continuing  to 
invest  in  legacy  systems  as  one  of  the  most  significant  national  defense 
challenges  for  the  21st  century.4  MDA  has  established  key  decision  points 
at  which  it  will  decide  whether  to  continue  to  invest  in  certain  elements  of 
the  integrated  system.  For  example,  one  key  decision  point  is  the  booster 
flight  test  for  the  Kinetic  Energy  Interceptor  element,  scheduled  for  2008, 
at  which  time  a  decision  will  be  made  on  the  program’s  future.  A  key 
decision  point  for  the  Airborne  Laser  is  the  lethal  shootdown 
demonstration  scheduled  for  2009.5  Finally,  in  2009,  MDA  plans  a  motor 
test  for  the  Standard  Missile-3  block  2A. 

In  March  2006,  MDA  issued  a  report  on  boost  and  ascent  phase  capabilities 
in  response  to  two  mandates — one  in  the  House  Appropriations 
Committee  Report  on  the  Defense  Appropriations  Bill  for  Fiscal  Year  2006 
and  one  in  the  National  Defense  Authorization  Act  for  Fiscal  Year  2006. 

The  House  Appropriations  Committee  directed  the  Secretary  of  Defense  to 
conduct  a  study  on  boost  and  ascent  phase  intercepts  including  an 
assessment  of  operational  capabilities,  quantities  of  operational  assets 
required  for  various  deployment  periods,  basing  options,  and  an 
assessment  of  life-cycle  costs.6  Life-cycle  costs  are  the  total  cost  to  the 
government  for  a  program  over  its  full  life,  consisting  of  research  and 
development,  production,  operations,  maintenance,  and  disposal  costs  and 
are  helpful  in  assessing  whether  a  system’s  cost  is  affordable.  The 
Department  of  Defense  Authorization  Act  for  Fiscal  Year  2006  required  the 
Secretary  of  Defense  to  conduct  an  assessment  of  U.S.  missile  defense 
programs  that  are  designed  to  provide  capability  against  threat  ballistic 
missiles  in  the  boost/ascent  phase.  The  purpose  of  the  assessment  was  to 
compare  and  contrast  capabilities,  asset  requirements,  and  the  costs  for 
making  the  boost  and  ascent  phase  programs  operational.7  MDA,  on  behalf 


3GAO,  Budget  Process:  Long-term  Focus  Is  Critical,  GAO-04-585T  (Washington,  D.C.:  Mar. 
23,  2004). 

4GAO,  21st  Century  Challenges:  Reexamining  the  Base  of  the  Federal  Government, 
GAO-05-325SP  (Washington,  D.C.:  February  2005). 

5Although  there  are  other  interim  decision  points  leading  up  to  the  lethal  shootdown 
demonstration,  MDA  officials  stated  that  the  2009  key  decision  point  is  one  where 
decisions  may  be  made  about  the  future  of  the  Airborne  Laser  program. 

6H.R.  Rep.  No.  109-119  (2005). 

7Pub.L.  No.  109-163,  §  231  (2005). 
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of  DOD,  prepared  one  report  to  satisfy  both  requirements  and  sent  the 
report  to  all  four  defense  committees  on  March  30,  2006.  According  to 
MDA  officials,  that  report  was  prepared  specifically  to  satisfy  the 
congressional  requirements,  not  for  use  at  any  key  decision  point. 

To  satisfy  a  direction  from  the  House  Appropriations  Committee  report, 
we  agreed  to  review  the  DOD  report  including  assessing  the  report’s 
methodologies,  assumptions,  completeness,  and  results.8  This  report  is  our 
assessment  of  the  March  2006  MDA  report  and  how  DOD  can  build  on  this 
information  to  support  future  key  decision  points.  Accordingly,  we 
assessed  the  extent  to  which  (1)  information  reported  by  DOD  includes 
analysis  of  technical  and  operational  issues  and  whether  any  additional 
information  is  needed  to  support  future  decision  making  and  (2)  DOD 
presented  cost  information  to  Congress  that  is  complete  and  transparent. 
To  assess  the  technical  and  operational  information,  we  analyzed  the 
extent  to  which  the  MDA  report  clearly  explained  technical  maturity, 
modeling  assumptions  and  their  effect  on  results,  and  involved 
stakeholders  in  analyses  of  operational  issues — steps  which  contribute  to 
a  sound  and  complete  quality  study  and  are  embodied  in  relevant  generally 
accepted  research  standards.9  To  assess  the  cost  information  in  the  March 
2006  report,  we  compared  MDA’s  cost  data  with  key  principles  compiled 
from  various  DOD  and  GAO  sources  that  describe  how  to  develop 
accurate  and  reliable  life-cycle  cost  estimates  which  are  to  include 
development,  production,  and  all  direct  and  indirect  operating  and  support 
costs.  We  discussed  the  data  and  results  of  this  comparison  with  DOD 
officials  and  identified  some  limitations,  which  we  discuss  in  this  report. 
We  conducted  our  work  between  June  2006  and  February  2007  in 
accordance  with  generally  accepted  government  auditing  standards.  See 
appendix  I  for  a  more  complete  description  of  our  scope  and 
methodology. 


8While  the  House  Appropriations  Committee  report  originally  directed  us  to  conduct  a  joint 
study  with  DOD,  we  agreed  with  the  Defense  Subcommittee,  House  Appropriations 
Committee  that,  in  lieu  of  a  joint  study,  we  would  assess  the  Missile  Defense  Agency’s 
March  2006  report  on  boost  and  ascent  phase  capabilities. 

9In  a  September  2006  report,  GAO  identified  frequently  occurring,  generally  accepted 
research  standards  that  are  relevant  for  defense  studies  and  define  a  sound  and  complete 
study.  GAO,  Defense  Transportation:  Study  Limitations  Raise  Questions  about  the 
Adequacy  and  Completeness  of  the  Mobility  Capabilities  Study  and  Report,  GAO-06-938 
(Washington,  D.C.:  Sept.  20,  2006).  Also,  see  app.  I  for  a  more  complete  description  of  our 
scope  and  methodology. 
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This  report  is  one  in  a  series  we  have  issued  on  ballistic  missile  defense 
(see  the  list  of  related  GAO  products  at  the  end  of  this  report).  Some  of 
these  reports  have  focused  on  assessments  of  program  goals  and  progress 
in  developing  each  element.  For  example,  our  March  2006  report  found 
that  MDA  made  progress  in  the  development  and  fielding  of  some 
elements  but  that  time  pressures  caused  MDA  to  stray  from  a  knowledge- 
based  acquisition  strategy.10  This  is  significant  because  straying  from  a 
knowledge-based  approach  opens  the  door  to  greater  cost  and 
performance  risks  by  not  having  the  right  information  available  for 
decision  makers  at  the  right  time.11  Another  report  focused  on  DOD’s 
planning  to  operate  BMDS.  We  reported  in  May  2006  that  DOD  has  not 
established  criteria  that  would  have  to  be  met  before  declaring  BMDS 
operational  and  that  the  Future  Years  Defense  Program  (FYDP),  a  major 
source  of  budget  information,  does  not  provide  complete  and  transparent 
data  on  ballistic  missile  defense  operational  costs.12 


Results  in  Brief 


MDA’s  2006  report  to  Congress  included  some  useful  technical  and 
operational  information  on  boost  and  ascent  phase  capabilities  but  the 
information  in  the  report  has  several  limitations.  Specifically,  MDA’s 
report  provided  some  key  information  on  potential  boost  and  ascent  phase 
capabilities  by  describing  these  elements,  listing  upcoming  decision 
points,  and  discussing  geographic  areas  where  these  elements  could 
intercept  missiles  shortly  after  launch.  However,  the  analysis  did  not 
involve  DOD  stakeholders  such  as  the  services  and  combatant  commands, 
which  will  have  a  key  role  in  operating  the  elements,  and  did  not  clearly 
explain  modeling  assumptions  and  their  effects  on  results  as  identified  by 
relevant  research  standards.  For  example,  the  report  assumed  that  each 
element  would  perform  as  desired  and  that  the  elements  could  be 
deployed  at  optimum  locations,  but  it  did  not  explain  how  each  element’s 
performance  would  change  if  developing  technologies  do  not  perform  as 
expected.  In  addition,  the  report  did  not  address  the  challenges  in 
establishing  bases  at  the  locations  cited  or  provide  information  on  the 


10GAO,  Defense  Acquisitions:  Missile  Defense  Agency  Fields  Initial  Capability  but  Falls 
Short  of  Original  Goals,  GAO-06-327  (Washington,  D.C.:  Mar.  15,  2006). 

nGAO,  Missile  Defense:  Knowledge-Based  Practices  Are  Being  Adopted,  but  Risks 
Remain,  GAO-03-441  (Washington,  D.C.:  Apr.  30,  2003). 

12GAO,  Defense  Management:  Actions  Needed  to  Improve  Operational  Planning  and 
Visibility  of  Costs  for  Ballistic  Missile  Defense,  GAO-06-473  (Washington,  D.C.:  May  31, 
2006). 
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quantity  of  each  element  required  for  various  deployment  periods.  Two 
key  reasons  for  these  limitations,  which  the  report  acknowledges,  are  that 
some  data  on  the  elements  are  limited  at  this  time,  and  operational 
concepts  that  discuss  operations  from  forward  locations  have  not  been 
fully  vetted  with  the  services  and  combatant  commands.  However,  moving 
forward,  DOD  has  an  opportunity  to  involve  stakeholders  in  analyzing 
operational  and  technical  issues  to  provide  more  complete  data  that  senior 
DOD  and  congressional  leaders  can  use  to  make  informed  program 
decisions  following  key  tests  in  2008  and  2009  for  the  Kinetic  Energy 
Interceptor  and  Airborne  Laser  programs.  To  provide  decision  makers 
with  information  to  support  boost  and  ascent  phase  program  decisions  at 
future  key  decision  points,  we  are  recommending  that  DOD  include  all 
DOD  stakeholders  in  developing  and  analyzing  operational  issues 
regarding  what  is  needed  to  support  operations  at  U.S.  bases  and  potential 
forward  locations,  provide  specific  information  on  the  technical  progress 
of  each  element,  and  use  the  results  of  these  analyses  at  each  key  decision 
point. 

The  Missile  Defense  Agency  provided  some  cost  estimates  for  developing 
and  fielding  each  of  the  boost  and  ascent  phase  capabilities  in  its  March 
2006  report  to  Congress;  however,  these  estimates  have  several  limitations 
and  will  require  refinement  before  they  can  serve  as  a  basis  for  DOD  and 
congressional  decision  makers  to  compare  the  costs  of  boost  and  ascent 
phase  alternatives.  We  compared  the  report’s  cost  estimates  with  various 
DOD  and  GAO  sources  that  describe  key  principles  for  developing 
accurate  and  reliable  life-cycle  cost  estimates.  Based  on  our  analysis,  we 
found  that  the  estimates  provided  in  the  report  did  not  include  all  cost 
categories,  including  costs  to  establish  and  sustain  operations  at  U.S. 
bases  and  at  forward  locations.  Also,  MDA  did  not  calculate  costs  based 
on  quantities  the  warfighter  would  need  to  conduct  the  mission,  or 
conduct  a  sensitivity  analysis  identifying  the  effects  of  cost  drivers. 
Although  the  report  acknowledges  that,  at  this  time,  there  is  uncertainty  in 
estimating  life-cycle  costs  for  these  elements,  it  did  not  fully  disclose  the 
limitations  of  the  cost  estimates.  DOD  can  significantly  improve  the 
completeness  of  and  confidence  in  its  cost  estimates  for  boost  and  ascent 
phase  capabilities  as  it  prepares  for  future  investment  and  budget 
decisions.  For  example,  although  MDA  did  not  have  the  cost  estimates  in 
its  March  2006  report  independently  verified  because  doing  so  would  have 
required  several  months,  MDA  officials  agree  that  independent 
verification,  which  is  another  key  principle  of  reliable  cost  estimates,  will 
be  critical  to  support  major  decision  points.  We  are  recommending  that 
DOD  prepare,  periodically  update,  and  independently  verify  life-cycle  cost 
estimates  for  boost  and  ascent  phase  elements  in  accordance  with  key 
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principles  for  developing  accurate  and  reliable  life-cycle  cost  estimates 
and  that  these  verified  estimates  be  used  for  making  investment  decisions 
and  be  reported  with  budget  requests. 

In  commenting  on  a  draft  of  this  report,  DOD  agreed  with  our 
recommendations  regarding  the  need  for  analysis  of  technical  progress 
and  operational  issues  to  support  key  boost  and  ascent  phase  element 
decision  points.  DOD  also  partially  concurred  that  an  independent  life- 
cycle  cost  estimate  may  be  needed  to  inform  some  key  decision  points  but 
said  that  they  may  not  be  needed  at  others.  However,  DOD  did  not  agree  to 
prepare  and  periodically  update  full  life-cycle  cost  estimates  for  each 
boost  and  ascent  phase  element  to  support  key  decision  points,  and  report 
independently  verified  life  cycle  cost  estimates  with  budget  requests  and 
FYDP  funding  plans.  In  its  comments,  DOD  stated  that  it  will  direct  an 
independent  evaluation  of  life-cycle  costs  if  circumstances  warrant  or  the 
Director  of  MDA  declares  an  element  mature  enough  to  provide  a 
militarily  useful  capability.  However,  if,  as  DOD’s  comments  suggest,  such 
costs  are  not  assessed  until  circumstances  warrant  or  MDA’s  Director 
makes  such  a  declaration,  these  costs  may  not  be  available  early  enough  to 
help  shape  important  program  and  investment  decisions  and  consider 
trade-offs  among  elements.  We  continue  to  believe  our  recommendation 
has  merit  because  the  development  of  life-cycle  cost  estimates  that  include 
potential  operations  and  support  costs  would  improve  the  information 
available  to  decision  makers  and  increase  accountability  for  key  decisions 
that  could  involve  billions  of  dollars  at  a  time  when  DOD  will  likely  face 
competing  demands  for  resources.  Finally,  DOD  did  not  agree  to  report 
independently  verified  life-cycle  cost  estimates  along  with  budget  requests 
and  FYDP  funding  plans  since  the  development  of  total  life-cycle  cost 
estimates  for  operationalized  BMDS  capabilities  require  agreement 
between  MDA  and  the  lead  military  department  on  the  roles  and 
responsibilities  for  fielded  missile  defense  capabilities.  However, 
independent  verification  allows  decision  makers  to  gauge  whether  the 
program  is  executable.  Also,  DOD  procedures  specify  that  when  cost 
estimates  are  presented  to  the  Office  of  the  Secretary  of  Defense’s  Cost 
Analysis  Improvement  Group,  life-cycle  cost  estimates  should  be 
compared  with  the  FYDP  and  differences  explained.  Therefore,  we 
continue  to  believe  that  our  recommendation  has  merit  because,  without 
an  independent  cost  estimate  that  can  be  compared  to  budget  requests  and 
FYDP  funding  plans,  congressional  decision  makers  may  not  have  all  the 
necessary  information  to  assess  the  full  extent  of  future  resource 
requirements  if  the  boost  and  ascent  phase  capabilities  go  forward,  or 
assess  the  completeness  of  budget  requests  and  FYDP  funding  plans. 
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Background 


In  2002,  the  Secretary  of  Defense  created  MDA  to  develop  an  integrated 
system  that  would  have  the  ability  to  intercept  incoming  missiles  in  all 
phases  of  their  flight.13  In  developing  BMDS,  MDA  is  using  an  incremental 
approach  to  field  militarily  useful  capabilities  as  they  become  available. 
MDA  plans  to  field  capabilities  in  2-year  blocks.  The  configuration  of  a 
given  block  is  intended  to  build  on  the  work  completed  in  previous  blocks. 
For  example,  Block  2006  is  intended  to  build  on  capabilities  developed  in 
Block  2004,  and  is  scheduled  to  field  capabilities  during  calendar  years 
2006-07. 

The  integrated  BMDS  is  comprised  of  various  elements,  three  of  which  are 
intended  to  intercept  threat  missiles  in  their  boost  or  ascent  phase.  Table  1 
below  describes  each  of  these  elements  and  shows  the  MDA  projected 
dates  for  key  decision  points,  initial  capability,  and  tested  operational 
capability. 


13The  boost  phase  is  the  period  immediately  after  launch  when  the  missile’s  booster  stages 
are  still  thrusting  and  typically  lasts  3-5  minutes  for  intercontinental  ballistic  missiles.  The 
ascent  phase  is  when  the  booster  stages  have  stopped  thrusting  and  dropped  away  leaving 
a  warhead  and  possible  decoys.  The  midcourse  phase,  lasting  for  about  20  minutes,  begins 
after  the  missile  has  stopped  accelerating  and  the  warhead  travels  through  space  on  a 
predictable  path.  The  final  or  terminal  phase  begins  when  the  warhead  reenters  the 
atmosphere  and  lasts  approximately  a  minute  or  less. 
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Table  1:  Ballistic  Missile  Defense  Boost  and  Ascent  Phase  Elements,  Key  Decision  Points,  and  Their  Planned  Capability 

Dates 

Boost/ascent  phase 
element 

Description 

Key  decision  points 

Initial  capability 

Tested  capability 

Standard  Missile-3,  block 
2Aa 

Aegis  Ballistic  Missile  Defense, 
which  employs  the  Standard 

Missile,  is  a  ship-based  system 
designed  to  destroy  medium, 
intermediate,  and  certain 
intercontinental  ballistic  missiles  in 
the  ascent  and  midcourse  phases. 

2009 — motor  test  for 
the  21 -inch  diameter 
missile 

Block  2012 

Block  2014 

Kinetic  Energy  Interceptor 
(KEI) — land-based 

KEI  is  being  designed  to  destroy 
medium,  intermediate,  and 
intercontinental  ballistic  missiles 
during  the  boost,  ascent,  and 
midcourse  phases  of  flight.  A  land- 
based  unit  may  consist  of  5 
launchers,  10  missiles,  and  fire 
control  and  communications 
equipment. 

2008 — booster  flight 
test 

Block  2014 

Block  to  be  decided 

Kinetic  Energy 

Interceptor — sea-based 

KEI  is  being  designed  to  destroy 
medium,  intermediate,  and 
intercontinental  ballistic  missiles 
during  the  boost,  ascent,  and 
midcourse  phases  of  flight.  DOD  is 
currently  studying  sea-based 
platform  alternatives  including 
surface  combatants,  submarines,  or 
a  new  platform. 

2007—  platform 
selection 

2008 —  booster  flight 
test 

Block  to  be  decided 

Block  to  be  decided 

Airborne  Laser  (ABL) 

ABL  is  an  air-based  missile  defense 
system  designed  to  destroy  all 
classes  of  ballistic  missiles  during 
the  boost  phase.  ABL  also  has 
onboard  sensor  capability  designed 
to  detect,  track,  and  transmit 
targeting  data. 

2009— lethal 

shootdown 

demonstration 

Block  2016 

Block  2018 

Source:  GAO  summary  of  DOD  information. 


“The  Standard  Missile  is  intended  to  be  used  on  BMDS  upgraded  Aegis  cruisers  and  destroyers. 
Other  versions  of  the  Standard  Missile-3  are  being  developed  to  intercept  threat  missiles  in  their 
midcourse  phase. 


During  the  past  year,  Congress  requested  additional  information  and 
analyses  on  the  boost  and  ascent  phase  elements  from  DOD.  Specifically, 
House  Report  109-119  on  the  Department  of  Defense  Appropriations  Bill 
for  Fiscal  Year  2006  directed  the  Secretary  of  Defense  to  conduct  a  study 
to  review  the  early  engagement  of  ballistic  missiles  to  include  boost  and 
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ascent  phase  intercepts  and  submit  the  report  to  the  congressional  defense 
committees.14  The  report  was  to  include,  but  not  be  limited  to 

•  an  assessment  of  the  operational  capabilities  of  systems  against 
ballistic  missiles  launched  from  North  Korea  or  a  location  in  the  Middle 
East  against  the  continental  United  States,  Alaska,  or  Hawaii; 

•  an  assessment  of  the  quantity  of  operational  assets  required  for 
deployment  periods  of  7  days,  30  days,  90  days,  and  1  year; 

•  basing  options;  and 

•  an  assessment  of  life-cycle  costs15  to  include  research  and  development 
efforts,  procurement,  deployment,  operating,  and  infrastructure  costs. 

In  addition,  the  National  Defense  Authorization  Act  for  Fiscal  Year  2006 
required  the  Secretary  of  Defense  to  assess  missile  defense  programs 
designed  to  provide  capability  against  threat  ballistic  missiles  in  the 
boost/ascent  phase  of  flight.  The  purpose  of  this  assessment  was  to 
compare  and  contrast 

•  capabilities  of  those  programs  (if  operational)  to  defeat  ballistic 
missiles  from  North  Korea  or  a  location  in  the  Middle  East  against  the 
continental  United  States,  Alaska,  or  Hawaii;  and 

•  asset  requirements  and  costs  for  those  programs  to  become 
operational  with  the  capabilities  referred  to  above. 

MDA,  on  behalf  of  DOD,  prepared  one  report  to  satisfy  both  of  the  above 
requirements  and  sent  the  report  to  all  four  defense  committees  on  March 
30,  2006.  The  report  included  technical,  operational,  and  cost  information 
for  each  of  the  three  boost  and  ascent  phase  BMDS  elements.  The 
remainder  of  this  report  discusses  our  assessment  of  the  MDA  report  and 
how  DOD  can  build  on  this  information  to  support  future  key  decision 
points. 


14H.R.  Rep.  No.  109-119  (2005). 

15For  most  major  acquisition  programs,  DOD  prepares  a  life-cycle  cost  estimate  that  is 
independently  verified  for  major  program  reviews.  A  life-cycle  cost  includes  all  costs 
associated  with  a  weapon  system’s  research  and  development,  investment,  which  includes 
military  construction,  operations  and  support,  and  disposal. 
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MDA’s  March  2006 
Report  Includes  Some 
Useful  Information 
but  Has  Several 
Limitations 


MDA’s  March  2006  report  to  Congress  included  some  useful  technical  and 
operational  information  on  boost  and  ascent  phase  capabilities.  However, 
the  information  in  the  report  has  several  limitations — such  as  not 
including  stakeholders  in  the  analysis  or  explaining  how  assumptions 
affect  results.  Moving  forward,  DOD  can  enhance  its  ability  to  make 
informed  decisions  at  future  key  decision  points  by  including  stakeholders 
DOD-wide  in  conducting  analyses  to  provide  complete  technical  and 
operational  information.  Otherwise,  senior  DOD  and  congressional 
decision  makers  may  be  limited  in  their  ability  to  effectively  assess  the 
technical  progress  and  operational  effects  of  proceeding  with  one  or  more 
boost  and  ascent  phase  element. 


Report  Provided  Some 
Basic  Information  about 
Boost  and  Ascent  Phase 
Elements 


The  March  2006  report  to  Congress  contained  some  useful  technical  and 
operational  information  for  Congress.  For  example,  the  report  included  a 
detailed  description  of  the  three  boost  and  ascent  phase  elements,  which 
could  be  useful  for  those  unfamiliar  with  these  elements.  Additionally,  the 
report  listed  upcoming  knowledge  points  where  DOD  will  review  the 
progress  MDA  has  made  toward  developing  each  of  the  boost  and  ascent 
phase  elements.  Further,  the  report  discussed  geographic  areas  where 
boost  and  ascent  phase  elements  could  intercept  missiles  shortly  after 
launch  based  on  desired  technical  capabilities.  Also,  MDA  used  a  model16 
to  assess  the  desired  capabilities  of  each  BMDS  element  for  the  March 
2006  report  to  Congress.  Further,  the  modeling  environment  was  used  for 
several  past  BMDS  analyses  and  the  results  were  benchmarked  against 
other  models.  Finally,  MDA  performed  a  sensitivity  analysis  that  compared 
how  the  results  in  the  modeling  changed  when  different  assumptions  for 
targets’  propellants,  ascent  times,  hardness  levels,  and  bum  times  were 
used.  To  provide  context,  the  report  explained  that  the  boost  and  ascent 
phase  elements  are  in  the  early  stages  of  development  and  that  the 
operational  concepts  are  not  yet  mature. 


MDA’s  Analyses  Have 
Limitations  but  DOD  Can 
Improve  Information  to 
Support  Future  Decisions 


The  information  in  the  March  2006  report  has  several  limitations  because 
the  analyses  did  not  involve  stakeholders  and  did  not  clearly  explain 
modeling  assumptions  and  their  effects  on  results  as  identified  by  relevant 
research  standards.  The  relevant  research  standards  and  our  prior  work 
have  shown  that  coordination  with  stakeholders  from  study  design 


16The  model,  called  WILMA,  supports  quantitative  evaluation  of  ballistic  missile  defense 
system  performance  in  terms  of  probabilities  of  success. 
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Data  Collection  and  Analysis 
for  the  March  2006  Report  Did 
Not  Involve  Key  DOD 
Stakeholders 


through  reporting,  and  clearly  explained  assumptions  and  their  effects  on 
results,  can  enable  DOD  officials  to  make  fully  informed  program 
decisions.  As  a  result,  the  March  2006  report  presents  an  incomplete 
picture  of  technical  capabilities,  such  as  development  challenges  to  be 
overcome  in  order  to  achieve  desired  performance,  and  it  does  not  clearly 
explain  the  effects  of  operational  assumptions,  such  as  basing  locations, 
asset  quantities,  and  base  support  requirements.  As  a  step  in  the  right 
direction,  MDA  stated  that  it  plans  to  develop  criteria  to  assess  the 
boost/ascent  phase  elements  at  major  decision  points  in  a  process 
involving  the  combatant  commands.  Although  MDA  officials  told  us  that 
they  consult  stakeholders  in  a  variety  of  forums  other  than  the  March  2006 
report,  they  did  not  clearly  state  whether  or  how  the  services  or  other 
DOD  stakeholders  would  be  involved  in  developing  criteria  for  key 
decision  points  or  the  extent  to  which  their  analyses  would  include 
information  on  technical  and  operational  issues. 

MDA’s  analyses  did  not  involve  soliciting  or  using  information  from  key 
DOD  stakeholders  such  as  the  services,  combatant  commands,  and  joint 
staff  from  study  design  through  reporting.  For  example,  officials  from  the 
Office  of  the  Secretary  of  Defense  for  Program  Analysis  and  Evaluation 
and  the  Defense  Intelligence  Agency  stated  there  were  areas  where 
additional  information  would  have  improved  the  fidelity  of  the  results. 
First,  the  officials  stated  that  there  is  uncertainty  that  the  boost  and  ascent 
phase  elements  would  achieve  their  desired  capabilities  within  the 
timeframe  stated  in  the  report.  Second,  officials  from  both  organizations 
stated  that  the  report  could  have  been  enhanced  by  presenting  different 
views  of  the  type  and  capability  of  threats  the  United  States  could  face  and 
when  these  threats  could  realistically  be  expected  to  be  used  by 
adversaries.  Third,  officials  from  the  Office  of  the  Secretary  of  Defense  for 
Program  Analysis  and  Evaluation  said  that  the  MDA  report  did  not 
distinguish  between  countermeasures  that  could  be  used  in  the  near  term 
and  countermeasures  that  may  be  more  difficult  to  implement.  MDA 
officials  said  that  they  worked  with  the  Office  of  the  Secretary  of  Defense 
for  Program  Analysis  and  Evaluation  in  conducting  analyses  before  they 
began  work  on  the  March  2006  report.  MDA  also  stated  that  it  discussed 
the  draft  March  2006  report  with  Office  of  the  Secretary  of  Defense  for 
Program  Analysis  and  Evaluation  officials  and  included  some  of  their 
comments  in  the  report’s  final  version.  However,  without  communication 
with  stakeholders  from  study  design  through  reporting,  MDA  may  not  have 
had  all  potential  inputs  that  could  have  affected  how  the  type,  capability, 
and  likelihood  of  countermeasures  to  the  boost  and  ascent  phase  elements 
were  presented  in  its  report. 
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Additionally,  MDA  did  not  solicit  information  from  the  services,  combatant 
commands,  or  Joint  Staff  regarding  operational  issues  that  could  have 
affected  information  about  basing  and  the  quantities  of  elements  that 
could  be  required  to  support  operations.  Although  the  elements  have  to  be 
located  in  close  proximity  to  their  intended  targets,  and  the  report 
discusses  placing  the  elements  at  specific  forward  overseas  locations,  the 
report  does  not  include  a  basing  analysis  explaining  what  would  need  to 
be  done  to  support  operations  at  these  locations.  Specifically,  the  report 
did  not  include  any  discussion  of  the  infrastructure  or  security/force 
protection  that  will  be  needed  for  the  BMDS  elements.  Although  the  report 
mentions  some  support  requirements — such  as  the  Airborne  Laser’s  need 
for  unique  maintenance  and  support  equipment  and  skilled  personnel  to 
maintain  the  laser — the  report  did  not  fully  explain  how  these  support 
requirements  would  be  determined,  who  would  provide  or  fund  them,  or 
explain  the  operational  effect  if  this  support  is  not  provided.  For  instance, 
without  an  adequate  forward  operating  location,  the  boost  and  ascent 
phase  elements  would  have  to  operate  from  much  further  away  which 
would  significantly  limit  the  time  an  element  is  in  close  proximity  to 
potential  targets.  Developing  such  information  with  the  services,  Joint 
Staff,  and  combatant  commands  could  provide  a  much  more  complete 
explanation  of  operational  issues  and  challenges.  The  services  typically 
perform  site  analyses  to  ascertain  what  support  is  needed  for  a  new 
weapon  system  at  either  a  U.S.  or  overseas  location.  This  comprehensive 
analysis  examines  a  range  of  issues  from  fire  protection  to  security,  to 
infrastructure,  to  roads  and  airfields.  In  addition,  U.S.  Strategic  Command 
and  service  officials  told  us  that  this  type  of  support  must  be  planned  for 
in  advance  when  adding  a  new  system  to  any  base,  either  in  the  United 
States  or  a  forward  location. 

MDA  also  did  not  involve  stakeholders  in  assessing  the  quantities  of  each 
element  for  deployment  periods  of  7  days,  30  days,  90  days,  and  1  year. 

The  report  stated  that  limited  data  exist  at  this  time  for  a  full  assessment 
of  this  issue,  and  service,  Joint  Staff,  and  MDA  officials  acknowledged  that 
the  quantities  of  each  element  used  in  the  report  are  MDA  assumed 
quantities.  Service,  Joint  Staff,  and  U.S.  Strategic  Command  officials  stated 
that  they  have  not  completed  analyses  to  assess  quantities  the  warfighters 
may  require.  We  understand  that  operational  concepts  will  continue  to 
evolve  and  could  affect  required  quantities.  However,  stakeholders  such  as 
the  services,  Joint  Staff,  or  combatant  commands  could  have  assisted 
MDA  in  assessing  potential  quantities  required  for  various  deployment 
periods.  In  addition,  MDA  did  not  solicit  information  from  the  services, 
Joint  Staff,  or  combatant  commands  to  determine  if  those  organizations 
were  conducting  force  structure  analyses  for  the  boost  and  ascent  phase 
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Assumptions  and  Their  Effects 
on  Data  Presented  in  the  MDA 
Report  Were  Not  Clearly 
Explained 


elements.  We  learned  that  the  Navy  had  done  a  preliminary  analysis  in  July 
2005  and  that  the  Joint  Staff  has  begun  a  capabilities  mix  study  and  both 
include,  in  part,  an  analysis  of  quantities.  Thus,  in  preparing  for  future 
decision  points,  MDA’s  analysis  could  be  strengthened  by  including 
stakeholders  to  leverage  other  analyses.  For  example,  MDA  could  have 
presented  a  range  of  scenarios  to  show  how  the  quantities  required  to 
intercept  adversary  missiles  could  vary  depending  upon  the  number  of 
sites  covered  and  whether  continuous,  near-continuous,  or  sporadic 
coverage  is  provided. 

The  March  2006  report  to  Congress  did  not  clearly  explain  the 
assumptions  used  in  the  modeling  of  the  BMDS  elements’  capabilities  and 
did  not  explain  the  effects  those  assumptions  may  have  had  on  the  results. 
First,  the  model  inputs  for  the  technical  analysis  assumed  desired  rather 
than  demonstrated  performance,  and  the  report  does  not  fully  explain 
challenges  in  maturing  technologies  or  how  these  performance  predictions 
could  change  if  the  technologies  are  not  developed  as  desired  or  assumed. 
For  example,  although  the  model  MDA  used  is  capable  of  showing 
different  results  based  on  different  performance  assumptions,  the  report 
did  not  explain  how  the  number  of  successful  intercepts  may  change  if 
less  than  100  percent  of  the  desired  technical  capabilities  are  developed  as 
envisioned.  Thus  the  results  represent  the  best  expected  outcome.  Second, 
the  report  does  not  explain  the  current  status  of  technical  development  or 
the  challenges  in  maturing  each  element’s  critical  technologies  as  desired 
or  assumed  in  the  report.  DOD  best  practices  define  Technology 
Readiness  Levels  on  a  scale  of  1-9,  and  state  which  level  should  be 
reached  to  progress  past  specific  program  decision  points.17  However,  the 
March  2006  report  does  not  explain  the  current  Technology  Readiness 
Level  for  any  of  the  boost  and  ascent  phase  elements’  critical  technologies 
or  the  extent  to  which  the  technology  has  to  mature  to  attain  the 
performance  assumed  in  the  report.  For  example,  the  report  does  not 
explain  that  some  of  the  technologies  for  the  Airborne  Laser  have  to 
improve  between  60  percent  and  80  percent  and  the  report  does  not 
discuss  any  of  the  challenges  MDA  faces  in  doing  so. 


^Technology  Readiness  Levels  are  measured  on  a  scale  of  1-9,  beginning  with  paper 
studies  of  a  technology’s  feasibility  (level  1)  and  culminating  with  a  technology  fully 
integrated  into  a  completed  product  (level  9). 
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DOD  Can 

Significantly  Improve 
Life-Cycle  Cost 
Estimates  to  Support 
Future  Investment 
and  Budget  Decisions 
for  Boost  and  Ascent 
Phase  Capabilities 


The  March  2006  report  to  Congress  provides  cost  estimates  for  each  of  the 
boost  and  ascent  phase  capabilities;  however,  the  cost  estimates  in  the 
report  have  several  limitations  that  raise  questions  about  their  usefulness. 
We  compared  the  report’s  cost  estimates  with  various  DOD  and  GAO 
sources  that  describe  key  principles  for  developing  accurate  and  reliable 
life-cycle  cost  estimates.18  Based  on  our  analysis,  we  found  that  MDA  did 
not  include  all  cost  categories,  calculate  costs  based  on  warfighter 
quantities,  and  did  not  conduct  a  sensitivity  analysis  to  assess  the  effects 
of  cost  drivers.  Moreover,  although  MDA’s  report  acknowledges 
uncertainty  in  the  cost  estimates,  the  report  does  not  fully  disclose  the 
limitations  of  the  cost  estimates.  DOD  can  significantly  improve  the 
completeness  of  and  confidence  in  cost  estimates  for  boost  and  ascent 
phase  capabilities  as  it  prepares  for  future  investment  and  budget 
decisions.  For  example,  although  DOD  did  not  have  its  cost  estimate  for  its 
March  2006  report  independently  verified  because  doing  so  would  have 
taken  several  months,  MDA  officials  agreed  that  independently  verified 
cost  estimates  will  be  critical  to  support  major  decision  points  for  boost 
and  ascent  phase  capabilities.19  In  addition,  as  these  capabilities  mature, 
MDA  officials  agreed  that  showing  cost  estimates  over  time  and 
conducting  uncertainty  analyses  will  be  needed  to  support  key  program 
and  investment  decisions. 


Cost  Estimates  Did  Not  The  cost  estimates  provided  in  the  MDA  report  included  some 
Include  All  Cost  Categories  development,  production,  and  operations/support  costs  for  each  boost  and 

ascent  phase  element  but  were  not  fully  developed  or  verified  according  to 
key  principles  for  developing  life-cycle  cost  estimates.  Life-cycle  costs  are 
the  total  cost  to  the  government  for  a  program  over  its  full  life,  including 


lsThe  criteria  for  developing  accurate  and  reliable  life-cycle  cost  estimates  were  compiled 
from  several  sources  including  the  DOD  Manual  5000.4-M  Cost  Analysis  Guidance  and 
Procedures,  the  Defense  Acquisition  Guidebook,  and  an  exposure  draft  of  GAO’s  Cost 
Assessment  Guide.  See  app.  I  for  a  more  complete  description  of  our  scope  and 
methodology. 

19As  we  reported  in  GAO-05-817,  independent  cost  estimates  are  normally  first  required  for 
Milestone  B,  the  decision  to  enter  system  development  and  demonstration.  GAO,  Defense 
Acquisi  tions:  Actions  needed  to  Ensure  Adequate  Funding  for  Operation  and 
Sustainment  of  the  Ballistic  Missile  Defense  System,  GAO-05-817  (Washington,  D.C.:  Sept. 
6,  2005).  However,  because  BMDS  elements  do  not  enter  DOD’s  acquisition  cycle  until 
Milestone  C,  the  requirements  under  Section  2434  of  Title  10  may  not  be  applied  to  the 
BMDS  until  the  transition  phase  that  leads  to  the  transfer  to  a  military  service.  However, 
developing  cost  estimates  using  the  key  principles  that  we  describe  for  developing  accurate 
and  reliable  life-cycle  cost  estimates  would  provide  useful  information  to  DOD  decision 
makers  and  Congress. 
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the  costs  of  research  and  development,  investment,  operating  and  support, 
and  disposal. 

Based  on  our  comparison  of  the  life-cycle  cost  estimates  in  the  report  with 
key  principles  for  developing  life-cycle  cost  estimates,  we  found  that  the 
estimates  were  incomplete  in  several  ways.  First,  the  cost  estimates  did 
not  include  all  cost  categories,  such  as  costs  to  establish  and  sustain 
operations  at  U.S.  bases.  Instead,  MDA  assumed  that  the  elements  would 
be  placed  at  existing  bases  with  sufficient  base  support,  infrastructure  and 
security;  however,  some  of  these  costs  such  as  infrastructure  could  be 
significant.  For  example,  an  MDA  planning  document  cited  about  $87 
million  for  infrastructure  costs  to  support  a  ground-based  BMDS  element 
(Terminal  High  Altitude  Area  Defense).  Army  officials  confirmed  that 
training  facilities,  missile  storage  buildings,  and  a  motor  pool  were  built  at 
a  U.S.  base  specifically  to  support  this  element  and  it  is  likely  that  similar 
infrastructure  would  be  needed  to  support  the  land-based  Kinetic  Energy 
Interceptor.  Additionally,  MDA’s  cost  estimates  did  not  include  costs  to 
establish  and  sustain  operations  at  forward  overseas  locations,  even 
though  the  report  states  that  the  elements  will  have  to  be  located  in  close 
proximity  to  their  targets,  and  the  operational  concepts  for  Kinetic  Energy 
Interceptor  and  Airborne  Laser,  although  in  early  development,  state  that 
these  elements  will  be  operated  from  forward  locations.  Again,  these  are 
important  factors  to  consider — the  Airborne  Laser  operational  concept 
and  the  MDA  report  acknowledge  that  unique  support  will  be  required  to 
support  operations  at  any  forward  location  for  the  Airborne  Laser  such  as 
chemical  facilities,  unique  ground  support  equipment,  and  maintenance. 
Service,  Joint  Staff,  and  U.S.  Strategic  Command  officials  also  said  that 
these  elements  would  have  to  be  located  forward  and  could  be  used  as  a 
strategic  deterrent  in  peacetime. 

Second,  the  production  and  operating  cost  estimates  were  not  based  on 
warfighter  quantities,  that  is,  quantities  of  each  element  that  the  services 
and  combatant  commands  may  require  to  provide  needed  coverage  of 
potential  targets.  MDA  assumed  a  certain  quantity  of  each  element.  For 
example,  MDA  officials  told  us  that  they  assumed  96  Standard  Missile-3 
block  2A  missiles  because,  at  the  time  MDA  prepared  the  report,  they 
planned  to  buy  96  block  1A  missiles  developed  to  intercept  short-range 
ballistic  missiles.  However,  MDA  did  not  solicit  input  from  the  services, 
Joint  Staff,  or  combatant  commands  on  whether  they  had  done  or  begun 
analyses  to  determine  element  quantities. 

Third,  MDA  did  not  conduct  a  sensitivity  analysis  to  identify  the  effects  of 
cost  drivers.  A  sensitivity  analysis  is  a  way  to  identify  risk  by 
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demonstrating  how  the  cost  estimates  would  change  in  response  to 
different  values  for  specific  cost  drivers.  Therefore,  a  sensitivity  analysis 
should  be  performed  when  developing  cost  estimates,  and  the  results 
should  be  documented  and  reported  to  decision  makers.  This  means,  for 
example,  that  MDA  could  have  computed  costs  with  and  without 
significant  categories  of  costs  such  as  forward  bases  to  identify  the  effect 
that  adding  forward  bases  would  have  on  operating  costs.  The  House 
Armed  Services  Committee  report  on  the  National  Defense  Authorization 
Bill  for  Fiscal  Year  2006  recognized  that  operational  capabilities  and  costs 
must  be  taken  into  account  when  making  decisions  on  future  funding 
support.  Finally,  the  cost  estimates  did  not  estimate  costs  over  time — a 
process  known  as  time  phasing — which  can  assist  decision  makers  with 
budgetary  decisions.  The  MDA  report  showed  an  annual  cost  estimate  but 
did  not  state  for  how  many  years  the  development,  production,  and 
operating  costs  may  be  incurred.  Although  MDA  officials  stated  they  did 
not  prepare  time-phased  cost  estimates  in  order  to  prepare  the  report  to 
Congress  in  a  timely  manner,  they  agreed  that  showing  cost  estimates  over 
time  would  be  important  information  to  support  investment  decisions  at 
key  decision  points. 


Cost  Estimates  Were  Not 
Assessed  or  Independently 
Verified  for  MDA’s  2006 
Report  but  MDA  Officials 
See  Need  to  Do  So  for  Key 
Decision  Points 


Key  principles  for  developing  life-cycle  cost  estimates  also  include  two 
steps  for  assessing  the  confidence  of  cost  estimates.  However,  MDA  did 
not  take  these  steps  to  assess  the  confidence  of  the  estimates  reported  in 
March  2006.  First,  the  Missile  Defense  Agency  did  not  conduct  a  risk 
analysis  to  assess  the  level  of  uncertainty  for  most  of  the  cost  estimates  in 
the  MDA  report.  Risk  and  uncertainty  refer  to  the  fact  that,  because  a  cost 
estimate  is  a  prediction  of  the  future,  it  is  likely  that  the  estimated  cost  will 
differ  from  the  actual  cost.  It  is  useful  to  perform  a  risk  analysis  to 
quantify  the  degree  of  uncertainty  in  the  estimates.  By  using  standard 
computer  simulation  techniques,  an  overall  level  of  uncertainty  can  be 
developed  for  cost  estimates.  In  contrast,  MDA  officials  told  us  that  they 
could  only  provide  a  judgmental  confidence  level  for  the  most  of  the  cost 
estimates.  Second,  MDA  did  not  have  the  cost  estimates  in  the  report 
verified  by  an  independent  organization  such  as  DOD’s  Cost  Analysis 
Improvement  Group  because  doing  so  would  have  taken  several  months. 
However,  MDA  officials  agreed  that  independent  verification  of  cost 
estimates  would  be  important  information  to  support  investment  decisions 
at  key  decision  points.  According  to  the  key  principles  that  we  have 
identified,  all  life-cycle  cost  estimates  should  be  independently  verified  to 
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assure  accuracy,  completeness,  and  reliability.20  MDA  has  recognized  the 
value  in  independently  developed  cost  estimates.  In  2003,  MDA  and  the 
Cost  Analysis  Improvement  Group  developed  a  memorandum  of 
understanding  that  said,  in  part,  the  Cost  Analysis  Improvement  Group 
would  develop  independent  cost  estimates  for  the  approved  BMDS  and  its 
elements  as  appropriate  during  development  in  anticipation  of  transition 
to  production,  but  MDA  officials  said  that  little  work  was  completed  under 
this  agreement,  which  has  expired. 

Developing  complete  cost  estimates  in  which  decision  makers  can  have 
confidence  is  important  since  life-cycle  cost  estimates  usually  form  the 
basis  for  investment  decisions  and  annual  budget  requests.  Specifically, 
life-cycle  cost  estimates  that  include  all  cost  categories,  show  costs  over 
time,  include  warfighter  quantities,  include  an  assessment  of  cost  drivers, 
and  are  independently  verified  are  important  because  accurate  life-cycle 
cost  estimates  can  be  used  in  formulating  funding  requests  contained  in 
the  President’s  Budget  and  DOD’s  future  funding  plan,  the  Future  Years 
Defense  Program  (FYDP)  submitted  to  Congress.  Therefore,  there  is  a 
need  for  DOD  to  provide  transparent  budget  and  cost  planning 
information  to  Congress.  In  May  2006,  GAO  reported  that  the  FYDP,  a 
major  source  of  budget  and  future  funding  plans,  does  not  provide 
complete  and  transparent  data  on  ballistic  missile  defense  operational 
costs  because  the  FYDP’s  structure  does  not  provide  a  way  to  identify  and 
aggregate  these  costs.  It  is  important  that  Congress  has  confidence  in 
boost  and  ascent  phase  estimates  because  Congress  has  indicated  that  it  is 
concerned  with  the  affordability  of  pursuing  both  the  Airborne  Laser  and 
Kinetic  Energy  Interceptor  programs  in  parallel  through  2008. 

As  we  reported  in  2003,  DOD  assumes  increased  investment  risk  by  not 
having  information  available  for  decision  makers  at  the  right  time,  and  the 
level  of  anticipated  spending  magnifies  this  risk.21  Otherwise,  senior  DOD 
and  congressional  decision  makers  may  be  limited  in  their  ability  to  assess 


20As  we  reported  in  GAO-05-817,  independent  cost  estimates  are  normally  first  required  for 
Milestone  B,  the  decision  to  enter  system  development  and  demonstration.  However, 
because  BMDS  elements  do  not  enter  DOD’s  acquisition  cycle  until  Milestone  C,  the 
requirements  under  Section  2434  of  Title  10  may  not  be  applied  to  the  BMDS  imtil  the 
transition  phase  that  leads  to  the  transfer  to  a  military  service.  However,  developing  cost 
estimates  using  the  key  principles  that  we  describe  for  developing  accurate  and  reliable 
life-cycle  cost  estimates  would  provide  useful  information  to  DOD  decision  makers  and 
Congress. 

Z1GAO-03-441. 
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the  relative  cost  of  the  elements  if  all  cost  categories  are  not  included  and 
cost  drivers  are  not  identified.  Considering  competing  demands,  this  could 
also  limit  Congress’s  ability  to  consider  investment  decisions  or  evaluate 
whether  continued  expenditures  are  warranted.  MDA  officials  stated  that, 
in  developing  the  cost  estimates  for  the  March  2006  report,  they  decided 
not  to  follow  some  of  the  key  principles  for  developing  life-cycle  cost 
estimates  such  as  time  phasing  and  independent  verification  of  the  cost 
estimates  in  order  to  complete  the  report  in  a  timely  manner.  However,  the 
officials  also  agreed  that  these  key  principles  are  important  in  developing 
complete,  accurate,  and  reliable  life-cycle  cost  estimates  for  supporting 
investment  decisions  at  key  decision  points.  Therefore,  in  the  future,  when 
preparing  cost  estimates  to  be  used  in  support  of  key  decision  points, 

MDA  could  provide  decision  makers  with  more  complete,  accurate,  and 
reliable  cost  estimates  by  better  adhering  to  key  principles  for  developing 
life-cycle  cost  estimates. 


Conclusions 


Our  review  of  MDA’s  March  2006  report  on  boost  and  ascent  phase 
elements  identified  a  number  limitations  but  helps  to  illuminate  the  kind  of 
information  that  DOD  and  congressional  decision  makers  will  need 
following  upcoming  tests  for  boost  and  ascent  phase  elements.  We 
recognize  that  the  March  2006  report  was  prepared  in  response  to 
congressional  direction  rather  than  to  support  program  decisions.  We  also 
recognize  that,  at  the  time  of  MDA’s  report,  these  elements  were  early  in 
their  development  and  information  was  incomplete  and  changing.  Thus, 
the  focus  of  our  analysis  was  to  identify  additional  information  that  could 
enhance  future  program  and  investment  decisions. 

In  particular,  the  House  Armed  Services  Committee  has  raised  questions 
about  the  affordability  of  pursuing  both  the  Kinetic  Energy  Interceptor  and 
the  Airborne  Laser  in  parallel  through  the  projected  knowledge  point 
demonstrations,22  which  are  now  scheduled  for  2008  and  2009  respectively. 
It  is  important  that  these  decisions  be  both  well-informed  and  transparent 
because  of  the  long-term  funding  consequences.  DOD  and  congressional 
decision  makers’  ability  to  assess  which  elements  can  be  fully  developed, 
integrated,  and  operated  relative  to  the  others  will  be  enhanced  if  they 
have  the  benefit  of  information  based  on  more  rigorous  analysis  than  that 
contained  in  MDA’s  March  2006  report.  Looking  forward,  as  DOD 
strengthens  its  analyses  to  support  future  key  decisions,  DOD  and 


22H.R.  Rep.  No.  109-452  (2006). 
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congressional  decision  makers  will  be  able  to  use  more  complete 
information  to  assess  force  structure,  basing,  support,  and  infrastructure 
requirements,  as  well  as  technical  maturity,  budget  requests,  and  FYDP 
spending  plans,  in  deciding  whether  or  not  to  continue  developing  one, 
two,  or  all  three  boost  and  ascent  phase  elements  and  in  what  quantities. 


Recommendations  for 
Executive  Action 


•  Include  all  DOD  stakeholders  (including  services,  combatant 
commands,  Joint  Staff)  in  developing  and  analyzing  operational  issues 
regarding  what  is  needed  to  support  operations  at  U.S.  bases  and 
potential  forward  locations,  including  basing  assessments,  force 
structure  and  quantity  requirements,  infrastructure,  security/force 
protection,  maintenance,  and  personnel. 

•  Provide  specific  information  on  the  technical  progress  of  each  element. 
Specifically,  the  analysis  should  explain  current  technical  maturity 
versus  desired  technical  maturity  and  capabilities  of  all  major 
components  and  subsystems,  reasonable  model  inputs  on  element 
performance,  and  provide  a  clear  explanation  of  assumptions  and  their 
effect  on  results. 

•  Use  the  results  of  these  analyses  at  each  key  decision  point. 

To  provide  decision  makers  with  complete  and  reliable  data  on  the  costs 
of  each  boost/ascent  phase  BMDS  element  to  enhance  investment  and 
budget  decisions,  we  recommend  that  the  Secretary  of  Defense  take  the 
following  actions: 

•  Direct  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  to  require  MDA  to  prepare  and — to  support  key  decision 
points — periodically  update  a  full  life-cycle  cost  estimate  for  each 
boost/ascent  phase  element,  in  accordance  with  key  principles  for 
developing  accurate  and  reliable  life-cycle  cost  estimates,  that  includes 
all  operational  costs,  including  costs  to  establish  and  sustain 
operations  at  U.S.  bases  and  forward  locations,  and  that  is  based  on 
warfighter  quantities,  includes  sensitivity  analyses,  and  reflects  time 
phasing. 


To  provide  decision  makers  with  information  that  enables  them  to  clearly 
understand  the  technical  progress  and  operational  implications  of  each 
boost  and  ascent  phase  element  and  make  fully  informed,  fact-based, 
program  decisions  at  future  key  decision  points,  we  recommend  that  the 
Secretary  of  Defense  direct  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  to  take  the  following  actions  to 
support  key  decision  points  for  the  BMDS  boost  and  ascent  phase 
elements: 
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•  Direct  an  independent  group,  such  as  the  Cost  Analysis  Improvement 
Group,  to  prepare  an  independent  life-cycle  cost  estimate  for  each 
capability  at  each  key  decision  point. 

•  Direct  MDA  and  services  to  report  independently  verified  life-cycle 
cost  estimates  along  with  budget  requests  and  FYDP  funding  plans  for 
each  boost/ascent  phase  element. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report,  DOD  agreed  with  our 
recommendations  regarding  the  need  for  analysis  of  technical  progress 
and  operational  issues  to  support  key  boost  and  ascent  phase  element 
decision  points.  DOD  also  agreed  that  an  independent  life-cycle  cost 
estimate  may  be  needed  to  inform  some  key  decision  points  while  they 
may  not  be  needed  at  other  decision  points.  However,  DOD  did  not  agree 
to  prepare  and  periodically  update  full  life-cycle  cost  estimates  for  each 
boost  and  ascent  phase  element  to  support  key  decision  points,  and  report 
independently  verified  life  cycle  cost  estimates  with  budget  requests  and 
FYDP  funding  plans.  As  discussed  below,  we  continue  to  believe  our 
recommendations  have  merit  and  that  DOD  should  take  the  additional 
actions  we  have  recommended  to  provide  a  rigorous  analytical  basis  for 
future  decisions,  enhance  the  transparency  of  its  analyses,  and  increase 
accountability  for  key  decisions  that  could  involve  billions  of  dollars.  The 
department’s  comments  are  reprinted  in  their  entirety  in  appendix  II. 

DOD  agreed  with  our  recommendations  that  all  DOD  stakeholders  be 
included  in  developing  and  analyzing  operational  issues,  that  specific 
information  on  technical  progress  be  provided  to  explain  current  versus 
desired  capabilities,  and  that  the  results  of  both  analyses  be  used  at  key 
decision  points.  DOD  stated  in  its  comments  that  officials  from  MDA,  the 
military  departments,  the  combatant  commanders,  and  other  organizations 
are  collaborating  to  develop  an  operational  BMDS.  Moreover,  the  annual 
BMDS  Transition  and  Transfer  Plan  is  coordinated  with  the  service 
secretaries  and  other  stakeholders  and  serves  as  a  repository  for  plans, 
agreements,  responsibilities,  authorities,  and  issues.  DOD  also  stated  that 
key  program  decisions  are  and  will  continue  to  be  informed  by  detailed 
technical  analysis,  including  assessment  of  element  technical  maturity. 
However,  DOD  did  not  clearly  explain  how  future  decision  making  will  be 
enhanced  or  how  analyses  of  operational  issues  will  be  conducted  if,  as  in 
the  case  of  the  Kinetic  Energy  Interceptor,  DOD  has  not  assigned  a  service 
responsibility  for  operating  the  element  once  it  is  developed.  We  continue 
to  believe  that  DOD  and  congressional  decision  makers  will  need  more 
complete  information  on  support  requirements  at  upcoming  decision 
points  as  well  as  a  clear  comparison  of  current  versus  desired  technical 
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capabilities  in  deciding  whether  or  not  to  continue  developing  one,  two,  or 
all  three  boost  and  ascent  phase  elements. 

Regarding  our  recommendations  to  improve  cost  estimates  used  to 
support  key  investment  decisions,  DOD  partially  concurred  that 
independent  life-cycle  cost  estimates  may  be  required  to  inform  some  key 
decision  points  but  stated  that  other  key  decision  points  may  not. 

However,  DOD  did  not  agree  that  it  should  routinely  prepare  and 
periodically  update  a  full  life-cycle  cost  estimate  for  each  boost  and  ascent 
phase  element.  DOD  said  that  it  continuously  assesses  all  aspects  of  its 
development  efforts  and  will  direct  an  independent  evaluation  of  life-cycle 
costs  for  boost  and  ascent  phase  elements  if  circumstances  warrant  or  if 
MDA’s  Director  declares  an  element  mature  enough  to  provide  a  militarily 
useful  capability.  However,  if,  as  DOD’s  comments  suggest,  such  costs  are 
not  assessed  until  circumstances  warrant  or  MDA’s  Director  declares  an 
element  mature  enough  to  provide  a  militarily  useful  capability,  these 
costs  may  not  be  available  early  enough  to  help  shape  important  program 
and  investment  decisions  and  consider  trade-offs  among  elements. 
Moreover,  DOD’s  Operating  and  Support  Cost  Estimating  Guide, 
published  by  the  Cost  Analysis  Improvement  Group,  states  that  when  the 
Cost  Analysis  Improvement  Group  assists  the  Office  of  the  Secretary  of 
Defense  components  in  their  review  of  program  costs,  one  purpose  is  to 
determine  whether  a  new  system  will  be  affordable  to  operate  and 
support.23  Therefore,  such  analysis  must  be  done  early  enough  to  provide 
cost  data  that  will  be  considered  in  making  a  decision  to  field,  produce,  or 
transition  an  element.  We  continue  to  believe  our  recommendation  has 
merit  because  the  development  of  life-cycle  cost  estimates  that  include 
potential  operations  and  support  costs  would  improve  the  information 
available  to  decision  makers  and  increase  accountability  for  key  decisions 
that  could  involve  billions  of  dollars  at  a  time  when  DOD  will  likely  face 
competing  demands  for  resources. 

Finally,  DOD  did  not  agree  to  report  independently  verified  life-cycle  cost 
estimates  along  with  budget  requests  and  FYDP  funding  plans  for  each 
boost  and  ascent  phase  element.  DOD  stated  that  operations  and  support 
segments  of  the  budget  are  organized  by  functional  area  rather  than  by 
weapon  system  and  are  dependent  on  operations  and  support  concepts  of 
the  employing  military  department.  DOD  further  stated  that  development 


"3DOD,  Office  of  the  Secretary  of  Defense  Cost  Analysis  Improvement  Group,  Operating 
and  Support  Cost  Estimating  Guide  (May  1992). 
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of  total  life-cycle  cost  estimates  for  operational  BMDS  capabilities  requires 
agreement  between  MDA  and  the  lead  military  department  on  roles  and 
responsibilities  for  fielded  BMDS  capabilities  that  transcend  the  annual 
transition  planning  cycle  but  serve  as  a  basis  for  budget  submittals.  We 
recently  reported  that  MDA  enjoys  flexibility  in  developing  BMDS  but  this 
flexibility  comes  at  the  cost  of  transparency  and  accountability.24  One 
purpose  of  cost  estimates  is  to  support  the  budget  process  by  providing 
estimates  of  the  funding  required  to  efficiently  execute  a  program.  Also, 
independent  verification  of  cost  estimates  allows  decision  makers  to 
gauge  whether  the  program  is  executable.  Thus,  cost  estimating  is  the 
basis  for  establishing  and  defending  budgets  and  is  at  the  heart  of  the 
affordability  issue.  This  principle  is  stated  in  DOD  procedures  which 
specify  that  when  cost  results  are  presented  to  the  Office  of  the  Secretary 
of  Defense  Cost  Analysis  Improvement  Group,  the  program  office- 
developed  life-cycle  cost  estimate  should  be  compared  with  the  FYDP  and 
differences  explained.26  Therefore,  we  continue  to  believe  that  our 
recommendation  has  merit  because,  without  an  independent  cost  estimate 
that  can  be  compared  to  budget  requests  and  FYDP  funding  plans, 
congressional  decision  makers  may  not  have  all  the  necessary  information 
to  assess  the  full  extent  of  future  resource  requirements  if  the  boost  and 
ascent  phase  capabilities  go  forward,  or  assess  the  completeness  of  the 
cost  estimates  that  are  in  the  budget  request  and  FYDP  funding  plans. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense;  the 
Commander,  U.S.  Strategic  Command;  the  Director,  Missile  Defense 
Agency;  Chairman,  the  Joint  Chiefs  of  Staff;  and  the  Chiefs  of  Staff  of  the 
Army,  Navy,  and  Air  Force.  We  will  make  copies  available  to  others  upon 
request.  In  addition,  the  report  will  be  available  at  no  charge  on  the  GAO 
Web  site  at  http://www.gao.gov. 


24GAO,  Defense  Acquisitions:  Missile  Defense  Acquisition  Strategy  Generates  Results  but 
Delivers  Less  at  a  Higher  Cost,  GAO-07-387  (Washington,  D.C.:  Mar.  15,  2007). 

2BDOD,  Assistant  Secretary  of  Defense  (Program  Analysis  and  Evaluation),  Cost  Analysis 
Guidance  and  Procedures,  DOD  5000.4-M  (December  1992). 
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If  you  or  your  staff  have  any  questions,  please  call  either  Janet  St.  Laurent 
on  (202)  512-4402  or  Paul  Francis  on  (202)  512-2811.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the 
last  page  of  this  report.  Staff  members  who  made  key  contributions  to  this 
report  are  listed  in  appendix  III. 


Janet  A.  St.  Laurent 
Director, 

Defense  Capabilities  and  Management 


Paul  L.  Francis 
Director, 

Acquisition,  Sourcing,  and  Management 
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List  of  Committees 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Minority  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Daniel  K.  Inouye 
Chairman 

The  Honorable  Ted  Stevens 
Ranking  Minority  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Ike  Skelton 
Chairman 

The  Honorable  Duncan  L.  Hunter 
Ranking  Minority  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  John  P.  Murtha 
Chairman 

The  Honorable  C.  W.  Bill  Young 
Ranking  Minority  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


During  this  review,  we  focused  on  assessing  the  analytical  approach  the 
Missile  Defense  Agency  (MDA)  used  to  develop  its  March  2006  report  to 
Congress,  as  well  as  the  methodology  for  developing  the  cost  estimates  for 
each  of  the  three  Ballistic  Missile  Defense  System  (BMDS)  boost  and 
ascent  phase  elements. 

To  assess  the  extent  to  which  the  Department  of  Defense  (DOD)  is 
developing  technical  and  operational  information  useful  for  oversight  and 
that  will  support  decision  making  at  key  points,  we  compared  the 
analytical  approach  DOD  used  to  develop  its  March  2006  report  with 
generally  accepted  research  standards  that  are  relevant  for  defense  studies 
such  as  this,  that  define  a  sound  and  complete  study,  and  that  cover  all 
phases  of  a  study — design,  execution,  and  presentation  of  results.  The 
following  were  our  sources  for  these  standards: 

•  GAO,  Government  Auditing  Standards:  2003  Revision,  GAO-03-673G 
(Washington,  D.C.:  June  2003); 

•  GAO,  Designing  Evaluations,  GAO/PEMD-IO.1.4  (Washington,  D.C.: 
March  1991); 

•  GAO,  Dimensions  of  Quality,  GAO/QTM-94-1  (Washington,  D.C.: 
February  2004); 

•  RAND  Corporation,  RAND  Standards  for  High-Quality  Research  and 
Analysis  (Santa  Monica,  Calif.:  June  2004); 

•  Air  Force,  Office  of  Aerospace  Studies,  Analysts  Handbook:  On 
Understanding  the  Nature  of  Analysis  (January  2000); 

•  Air  Force,  Office  of  Aerospace  Studies,  Air  Force  Analysis  Handbook, 

A  Guide  for  Performing  Analysis  Studies:  For  Analysis  of  Alternatives 
or  Functional  Solution  Analysis  (July  2004); 

•  Department  of  Defense,  DOD  Modeling  and  Simulation  (M&S) 
Verification,  Validation,  Accreditation  (W&A),  Instruction  6000.61 
(Washington,  D.C.:  May  2003); 

•  Department  of  Defense,  Data  Collection,  Development,  and 
Management  in  Support  of  Strategic  Analysis,  Directive  8260.1 
(Washington,  D.C.:  Dec.  2,  2003);  and 

•  Department  of  Defense,  Implementation  of  Data  Collection, 
Development,  and  Management  for  Strategic  Analyses,  Instruction 
8260.2  (Washington,  D.C.:  Jan.  21,  2003). 
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For  a  more  complete  description  of  these  standards  and  how  we  identified 
them,  see  GAO-06-938,  appendix  I.1  In  applying  these  standards,  we 
focused  on  the  extent  to  which  stakeholders  were  involved  in  study  design 
and  analysis  as  well  as  the  extent  to  which  assumptions  were  reasonable 
and  their  effects  on  results  were  clearly  explained.  We  assessed  MDA 
briefings  that  explained  the  modeling  used  for  the  technical  analysis 
projecting  the  elements’  capabilities.  To  assess  the  basis  for  the  assumed 
performance  parameters  used  to  model  each  element’s  performance,  we 
traced  and  verified  a  nonprobability  sample  of  these  parameters  to  their 
source  documentation  and  concluded  that  they  were  generally  supported. 
To  evaluate  the  DOD  report’s  characterization  of  threats,  we  reviewed 
Defense  Intelligence  Agency  documents  and  discussed  the  type  and 
capability  of  threats  and  expected  BMDS  capabilities  with  officials  from 
the  Office  of  the  Secretary  of  Defense  for  Program  Analysis  and 
Evaluation  and  the  Defense  Intelligence  Agency.  In  addition,  to  gain  an 
understanding  of  the  extent  to  which  DOD  has  assessed  warfighter 
quantities  for  the  boost  and  ascent  phase  elements,  the  development  of 
operational  concepts,  and  operational  implications  of  employing  the  boost 
and  ascent  phase  elements  at  forward  locations,  we  evaluated  DOD  and 
service  guidance  on  assessing  sites  and  support  for  new  weapon  systems 
and  discussed  these  issues  with  officials  from  the  Joint  Staff;  U.S.  Army 
Headquarters  and  Space  and  Missile  Defense  Command;  U.S.  Strategic 
Command;  the  office  of  the  Chief  of  Naval  Operations  Surface  Warfare 
Directorate,  Ballistic  Missile  Defense  Division;  Air  Combat  Command;  and 
the  office  of  the  Secretary  of  the  Air  Force  for  Acquisition,  Global  Power 
Directorate.  Finally,  we  discussed  the  results  of  all  our  analyses  with 
officials  in  the  Joint  Staff;  U.S.  Strategic  Command;  the  Army’s  Space  and 
Missile  Defense  Command;  Office  of  the  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics;  Missile  Defense  Agency;  the  office 
of  the  Chief  of  Naval  Operations  Surface  Warfare  Directorate,  Ballistic 
Missile  Defense  Division;  the  office  of  the  Secretary  of  the  Air  Force  for 
Acquisition  Global  Power  Directorate;  and  Air  Combat  Command. 

To  assess  the  extent  to  which  DOD  presented  cost  information  to 
Congress  that  is  complete  and  transparent,  we  first  assessed  how  MDA 
developed  its  estimates  and  then  compared  the  method  by  which  those 
estimates  were  prepared  to  key  principles  compiled  from  various  DOD  and 


^AO,  Defense  Transportation:  Study  Limitations  Raise  Questions  about  the  Adequacy 
and  Completeness  of  the  Mobility  Capabilities  Study  and  Report,  GAO-06-938 
(Washington,  D.C.:  Sept.  20,  2006). 
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GAO  sources  that  describe  how  to  develop  accurate  and  reliable  life-cycle 
cost  estimates  to  determine  their  completeness  and  the  extent  to  which 
DOD  took  steps  to  assess  confidence  in  the  estimates.  The  following  were 
our  sources  for  compiling  the  cost  criteria: 

•  Department  of  Defense,  Assistant  Secretary  of  Defense  (Program 
Analysis  and  Evaluation),  Cost  Analysis  Guidance  and  Procedures, 

DOD  Manual  5000.4-M  (December  1992); 

•  Department  of  Defense,  Office  of  the  Secretary  of  Defense  Cost 
Analysis  Improvement  Group,  Operating  and  Support  Cost  Estimating 
Guide  (May  1992); 

•  Department  of  Defense,  Defense  Acquisition  University,  Defense 
Acquisition  Guidebook  (online  at  http://akss.dau.mil/dag); 

•  Department  of  Defense,  Defense  Acquisition  University,  Introduction 
to  Cost  Analysis  (April  2006); 

•  Air  Force,  Office  of  Aerospace  Studies,  Air  Force  Analysis  Handbook: 

A  Guide  for  Performing  Analysis  Studies  for  Analysis  of  Alternatives  or 
Functional  Solution  Analysis  (July  2004); 

•  Air  Force,  Base  Support  and  Expeditionary  Site  Planning,  Air  Force 
Instruction  10-404  (March  2004);  and 

•  GAO,  GAO  Cost  Assessment  Guide  (currently  under  development). 

In  addition,  we  met  with  DOD  officials  from  MDA,  U.S.  Strategic 
Command,  the  Joint  Staff,  Army,  Navy  and  Air  Force  to  determine  the 
extent  to  which  they  were  involved  in  developing  the  cost  estimates  for 
the  DOD  report.  Finally,  we  corroborated  our  methodology  and  results 
with  officials  from  the  Office  of  the  Under  Secretary  of  Defense,  Program, 
Analysis  and  Evaluation  (Cost  Analysis  Improvement  Group)  and  the 
Office  of  the  Under  Secretary  of  Defense  (Comptroller)  and  they  agreed 
that  our  methodology  for  examining  the  report’s  cost  estimates  was 
reasonable  and  consistent  with  key  principles  for  developing  accurate  and 
reliable  life-cycle  cost  estimates.  We  identified  some  data  limitations  with 
the  cost  estimates  which  we  discuss  in  this  report. 

We  provided  a  draft  of  this  report  to  DOD  for  its  review  and  incorporated 
its  comments  where  appropriate.  Our  review  was  conducted  between  June 
2006  and  February  2007  in  accordance  with  generally  accepted 
government  auditing  standards. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION, 
TECHNOLOGY 
AND  LOGISTICS 


MAR  2  8  2007 


Ms.  Janet  St.  Laurent 

Director,  Defense  Capabilities  and  Management 
U.  S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 


Dear  Ms.  St.  Laurent: 


This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report 
GAO-07-430,  “MISSILE  DEFENSE:  Actions  Needed  to  Improve  Information  for 
Supporting  Future  Key  Decisions  for  Boost  and  Ascent  Phase  Elements,”  dated 
February  28,  2007  (GAO  Code  350861). 

The  DoD  concurs  with  three,  partially  concurs  with  one,  and  non-concurs  with  two 
of  the  draft  report’s  recommendations.  The  rationale  for  our  position  is  included  in  the 
enclosure. 

We  appreciate  the  opportunity  to  comment  on  the  draft  report.  My  point  of 
contact  for  this  effort  is  Mr.  David  Crim,  (703)  697-5385,  David.Crim@osd.mil 

Sincerely, 


Director 

Portfolio  Systems  Acquisition 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  -  DATED  FEBRUARY  28,  2007 
GAO  CODE  350861/GAO-07-430 

“MISSILE  DEFENSE:  Actions  Needed  to  Improve  Information 
for  Supporting  Future  Key  Decisions  for  Boost  and  Ascent  Phase  Elements” 


DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  RECOMMENDATION 


RECOMMENDATION  1:  The  GAO  recommended  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  to  include  all  DoD 
stakeholders  (including  services,  combatant  commands.  Joint  Staff)  in  developing  and 
analyzing  operational  issues  regarding  what  is  needed  to  support  operations  at  U.S.  bases  and 
potential  forward  locations,  including  basing  assessments,  force  structure  and  quantity 
requirements,  infrastructure,  security/force  protection;  maintenance,  and  personnel,  for  the 
Ballistic  Missile  Defense  System  (BMDS)  boost  and  ascent  phase  elements.  (Page  21/GAO 
Draft  Report) 

DoD  RESPONSE:  Concur.  As  reported  in  the  DoD  Report  to  Congress  on  Transition  of 
Ballistic  Missile  Defense  Programs  to  the  Military  Departments  for  2007,  the  DoD  has  made 
significant  progress  in  planning  for  the  transition  of  operating  and  support  responsibilities  and 
authorities  for  elements  of  the  BMDS  to  the  Military  Departments.  Members  of  the  Missile 
Defense  Agency  (MDA),  the  Military  Departments,  the  Combatant  Commands,  the  Joint  Staff, 
and  the  Office  of  the  Secretary  of  Defense  are  collaborating  by  way  of  an  Integrated  Product 
Team  (IPT)  to  operationalize  the  BMDS.  This  Transition  and  Transfer  IPT  produces  an 
annual  DoD  document  referred  to  as  the  BMDS  Transition  and  Transfer  Plan  (T&TP)  that 
serves  as  a  repository  for  plans,  agreements,  responsibilities,  authorities,  and  issues.  The 
T&TP  is  submitted  by  MDA  to  USD(AT&L)  annually  on  March  1,  and  then  coordinated  with 
Service  Secretaries  and  other  stakeholders,  and,  with  regards  to  operationalizing  the  BMDS, 
serves  as  the  basis  for  MDA  and  Service  budget  formulation. 


RECOMMENDATION  2:  The  GAO  recommended  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  to  provide  specific 
information  on  the  technical  progress  of  each  BMDS  boost  and  ascent  phase  element  to  support 
key  decision  points.  Specifically,  the  analysis  should  explain  current  technical  maturity  of  all 
major  components  and  subsystems  in  comparison  to  desired  technical  maturity,  reasonable 
model  inputs  on  element  performance,  and  a  clear  explanation  of  assumptions  and  their  effect 
on  results.  (Page  21 /GAO  Draft  Report) 

DoD  RESPONSE:  Concur.  Key  program  decisions  are  and  will  continue  to  be  informed  by 
detailed  technical  analysis,  including  assessments  of  element  technical  maturity,  and  any 
sensitivity  of  element  performance  estimates  to  the  models  and  simulations  used. 
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RECOMMENDATION  3:  The  GAO  recommended  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  to  use  the  results  of 
these  analyses  at  each  key  decision  point  for  the  BMDS  boost  and  ascent  phase  elements. 

(Page  21  /GAO  Draft  Report) 

DoD  RESPONSE:  Concur.  Key  program  decisions  are  and  will  continue  to  be  informed  by 
detailed  technical  analysis,  including  assessments  of  element  technical  maturity,  and  any 
sensitivity  of  element  performance  estimates  to  the  models  and  simulations  used. 

RECOMMENDATION  4:  The  GAO  recommended  that  the  Secretary  of  Defense  direct  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  to  require  the  Missile 
Defense  Agency  (MDA)  to  prepare  and  periodically  update  a  full  life  cycle  cost  estimate  for 
each  boost/ascent  phase  element,  in  accordance  with  key  principles  for  developing  accurate  and 
reliable  life  cycle  cost  estimates,  that  includes  all  operational  costs,  including  costs  to  establish 
and  sustain  operations  at  U.S.  bases  and  forward  locations,  is  based  on  warfighter  quantities, 
includes  sensitivity  analyses,  and  reflects  time  phasing.  (Pages  21-22/GAO  Draft  Report) 

DoD  RESPONSE:  Non-concur.  As  noted  in  the  DoD  Report  to  Congress  on  Transition  of 
Ballistic  Missile  Defense  Programs  to  the  Military  Departments  for  2007,  DoD  guidance 
directs  that  BMDS  elements  be  managed  in  three  phases:  development;  transition,  and 
procurement  and  operations.  The  Director,  MDA  is  to  manage  the  BMDS  through  the 
development  and  transition  phases,  baseline  the  capability  and  configuration  of  each  element 
and  capability  block,  and  define  interoperability  standards.  The  Military  Departments  provide 
forces,  as  needed,  to  support  fielding  of  early  and  or  contingency  capability.  During  early 
fielding  and  the  transition  phase,  responsibilities  are  shared  by  MDA  and  the  Military 
Departments;  specific  responsibilities  and  authorities  are  documented  in  the  T&TP,  agreed  to 
by  MDA  and  the  Military  Departments,  and  subsequently  programmed  for  by  the  applicable 
Department  or  Agency.  Beyond  early  fielding  and  or  transition,  and  after  an  appropriate 
milestone  decision,  the  Military  Departments  will  provide  resources  to  procure  and  support  the 
transitioned  elements  as  programs  in  their  own  right  and  as  reflected  by  the  planned  force 
structure.  DoD  continuously  assesses  all  aspects  of  its  developments  and  will  direct  an 
independent  evaluation  of  life  cycle  costs  for  boost  and  ascent  phase  elements  if  circumstances 
warrant  or  the  Director,  MDA  declares  an  element  mature  enough  to  provide  a  militarily 
useful  capability.  In  making  this  declaration,  the  Director,  MDA  will  rely  on  all  data 
available,  including  available  cost  estimates.  These  estimates  will  be  refined  in  conjunction 
with  designated  lead  Military  Departments  as  part  of  the  Transition  and  Transfer  IPT  and 
documented  in  the  T&TP  plan. 

RECOMMENDATION  5:  The  GAO  recommended  that  the  Secretary  of  Defense  direct  an 
independent  group,  such  as  the  Cost  Analysis  Improvement  Group,  to  prepare  an  independent 
Life  Cycle  Cost  Estimate  for  each  capability  at  each  key  decision  point.  Pages  21-22/GAO 
Draft  Report) 
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DoD  RESPONSE:  Partially  concur.  While  some  of  the  key  decision  points  may  require 
independent  life  cycle  cost  estimates,  others  may  not.  As  noted  in  the  DoD  Response  to 
Recommendation  4  above,  the  Director,  MDA  may  declare  an  element  mature  enough  to 
provide  a  militarily  useful  capability.  Based  upon  this  recommendation,  DoD  will  determine  if 
that  capability  is  ready  to  begin  early  fielding  or  proceed  to  a  milestone  review.  DoD 
stakeholders  will  rely  on  independent  life  cycle  cost  estimates  to  inform  these  and  other  key 
decisions  as  applicable. 


RECOMMENDATION  6:  The  GAO  recommended  that  the  Secretary  of  Defense  direct 
MDA  and  the  Military  Services  to  report  independently  verified  life  cycle  cost  estimates  along 
with  budget  requests  and  Future  Years  Defense  Plan  (FYDP)  funding  plans  for  each 
boost/ascent  phase  element.  (Pages  21-22/GAO  Draft  Report) 

DoD  RESPONSE:  Non-concur.  Operations  and  support  segments  of  the  budget  are 
organized  by  functional  area  rather  than  by  weapon  system.  The  functional  areas  are 
characterized  by  personnel,  unit-level  consumption,  various  levels  of  maintenance,  sustaining 
support,  and  indirect  support  costs.  Although  the  costs  of  certain  elements  (such  as  personnel) 
can  often  be  identified  explicitly,  other  elements  may  be  common  to  more  than  one  weapon 
system  and  are  dependent  on  operations  and  support  concepts  of  the  employing  Military 
Department.  The  cost  of  these  latter  elements  may  be  allocated  among  respective  systems  and 
support  organizations.  Development  of  total  life  cycle  cost  estimates  for  operationalized 
BMDS  capabilities  requires  agreement  between  MDA  and  lead  Military  Departments  on  roles 
and  responsibilities  for  fielded  missile  defense  capabilities.  The  T&TP,  updated  annually  or 
as  significant  changes  warrant,  constitutes  a  “snapshot”  of  a  work-in-progress  intended  to 
capture  current  BMDS  capability  transition  status,  roles,  responsibilities,  and  future 
transition  plans  for  the  components  and  elements  of  the  BMDS.  The  annual  snapshot  is 
taken  with  the  intention  of  clarifying  roles  and  responsibilities  and  documenting 
agreements  between  the  Military  Departments  and  MDA.  These  agreements  cover 
responsibilities,  deliverables,  funding,  and  schedules  that  transcend  the  annual  BMDS 
transition  plan  cycle.  The  T&TP  is  submitted  early  enough  in  the  Programming  Planning 
and  Budgeting  Cycle  so  that  it  can  be  used  by  MDA  and  the  Services  as  a  basis  for  their 
BMDS  budget  submittals  and  to  provide  needed  insights  into  estimated  life  cycle  costs  of 
BMDS  capabilities. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 
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